Skip to content

feat: transcript language picker (23 languages) with script-aware PDFs - #8

Merged
Adeolu05 merged 1 commit into
fix/shared-limits-and-log-privacyfrom
feat/language-picker
Sep 28, 2026
Merged

Adeolu05 merged 1 commit into
fix/shared-limits-and-log-privacyfrom
feat/language-picker

Conversation

@Adeolu05

@Adeolu05 Adeolu05 commented Sep 28, 2026 •

Copy link
Copy Markdown
Owner

Stacked on #7 (merge order: #5, #6, #7, then this).

Summary

Language picker

  • Web: a "Transcript language" select sits under the URL field, and the choice is remembered per browser. The list comes from GET /api/v1/config, so web, API and bot share one source (app/core/languages.py, 23 languages).
  • Telegram: a new /language command and a Language button on /start. The choice is saved with the other preferences, and the ready message shows the language delivered.
  • YouTube: uses a native track in the target language if one exists, otherwise auto-translates. Vimeo can't translate, so it picks the uploader's matching track, then English, then any track.
  • The response says what actually came back: language/language_name, source_language_name, translated and language_fallback. The UI shows either "Auto-translated by YouTube from English…" or "Japanese captions weren't available, so this is in English."
  • zh-Hans and zh-Hant are now treated as different languages (before, only the base code zh was compared). The transcript cache key includes the language.

Bug found in a live test: translation errors were caught by a bare except Exception, so an IP block or timeout during translation quietly returned the original-language captions. The UI would call them "not available", and the result was cached for 72h. Now only NotTranslatable/TranslationLanguageNotAvailable fall back. Anything else surfaces as an error and nothing is cached.

PDF fonts (existing bug, now much more visible)
Every PDF used plain Helvetica, which only covers Western European characters. Polish, Vietnamese, Russian or Japanese text came out as empty boxes. The font is now chosen from the text itself:

Text Font
Western European Helvetica (output unchanged)
Latin-extended, Cyrillic, Greek embedded DejaVu Sans (fonts-dejavu-core in Docker)
Chinese, Japanese ReportLab built-in CID fonts, CJK line wrapping
Korean embedded Nanum Gothic (fonts-nanum). ReportLab's Korean CID fonts render blank in PDFium (Chrome's viewer).
Arabic, Hebrew, Indic, Thai refused with "PDF isn't available for this language yet". ReportLab can't shape these. DOCX/TXT/SRT/VTT still work. The web PDF button is disabled with an explanation, and the bot falls back to DOCX.

Only letters decide whether a PDF can be built. Stray symbols a font lacks (emoji, ♪) are dropped, so they never block a PDF.

Test plan

  • 99 unit tests pass (21 new): matching rules, native vs translated vs fallback fetch, the zh script split, blocked translation raising instead of falling back, cache key, API fields, the Arabic PDF refusal + DOCX success, font selection
  • Rendered sample PDFs with PDFium: Polish, Vietnamese, Russian, Greek, Japanese (wrapped), Korean, Simplified and Traditional Chinese all display correctly
  • Live YouTube: a video with non-translatable tracks correctly fell back to English (3.4s). I stopped live testing when YouTube started rate-limiting my IP.
  • Live: extract a video in a translated language once the IP limit clears
  • Docker build: check fonts-dejavu-core + fonts-nanum install, and a Russian/Korean PDF from the container
  • Telegram: /language → pick → send a link

- Web picker + Telegram /language: choose the transcript language; YouTube
  uses a native track or auto-translates, Vimeo picks the matching uploaded
  track. Responses say what came back (translated / fallback) and the UI
  explains it. Language list served from /api/v1/config.
- Correct Chinese handling: zh-Hans and zh-Hant no longer treated as equal.
- Cache key includes the language.
- Translation failures from blocks/timeouts now surface as errors instead of
  silently returning (and caching) original-language captions.
- PDFs: fonts chosen from the actual text. Embedded DejaVu Sans for
  Latin-extended/Cyrillic/Greek, built-in CID fonts for Chinese/Japanese,
  embedded Nanum Gothic for Korean; Arabic/Hebrew/Indic/Thai get a clear
  "PDF not available" (DOCX/TXT still work) instead of broken glyphs.
  Previously every PDF used Helvetica, so e.g. Polish or Russian text came
  out as empty boxes.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@vercel

vercel Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
transcriptflow Ready Ready Preview Sep 28, 2026 6:11pm UTC

@Adeolu05
Adeolu05 merged commit da2fc27 into fix/shared-limits-and-log-privacy Sep 28, 2026
4 checks passed
Adeolu05 added a commit that referenced this pull request Oct 13, 2026

This branch was successfully deployed

1 active deployment
Preview — 443c6154 Deployed Sep 28, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant